EAML: ensemble self-attention-based mutual learning network for document image classification

In the recent past, complex deep neural networks have received huge interest in various document understanding tasks such as image classification and retrieval. As many types a distinct visual style, learning only features with CNNs to classify images has encountered problem of low inter-class discrimination, high intra-class structural variations between its categories. parallel, text-level jointly learned corresponding properties within given considerably improved performance terms accuracy. this paper, we design self-attention-based fusion module that serves block our ensemble trainable network. It allows simultaneously learn discriminant text modalities throughout training stage. Besides, encourage mutual by transferring positive knowledge during This constraint is realized adding truncated Kullback–Leibler divergence loss (Tr- $$\hbox {KLD}_{{\mathrm{Reg}}}$$ ) new regularization term, conventional supervised setting. To best knowledge, first time leverage approach along perform classification. The experimental results illustrate effectiveness accuracy for single-modal multi-modal modalities. Thus, proposed model outperforms state-of-the-art based on benchmark RVL-CDIP Tobacco-3482 datasets.